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SECTION I 
INTRODUCTION 

The Management Information and Instructional Systems Activity (MIISA) 
at Memphis currently provides computer services to the Navy Computer Managed 
Instruction (CMI) system and to numerous non-CMI users. There is concern 
about the impact the non-CMI users are having on the ability of the system 
to meet the CMI requirements as well as the effect of computer downtime on 
training time. There is also a need to identify other factors which degrade, 
or have the potential to degrade, CMI system performance. According y, the 
Chief of Naval Education and Training (CNET) tasked the Training Analysis 
and Evaluation Group (TAEG)i to identify^and evaluate those factors which 
adversely impact CMI response time.l' 

PURPOSE OF THIS STUDY 

The purpose of this study was to provide information which could be 
used to maintain system reliabiity as the CMI processing requirements 
increase and to provide data to support expanding the CMI system capability 
to serve an anticipated increase in the student load. ' 

Specific objectives of the study were to: 

. analyze the response time, interruptions, and availability of the 
CMI system for the period between March and October 1981 

. identify the non-CMI users, summarize the requirements these users 
are placing on the system, "and determine the impact these users 
are having on the ability of the system to respond to CMI require- 
ments 

• identify the hardware/software limitations of the present CMI 
system and explore the possibility for improving both hardware and 
software to increase efficiency, capability, and reliability of 
the CMI system 

• analyze the relationship between computer downtime and lost train- 
ing time to see if computer unavailability extends training time. 

APPROACH 

An analytical study was done which utilized data obtained from the 
following sources: 

• Computer terminal user reports were collected weekly between March 
to October 1981 and served as one basis for assessing the CMI 
system performance. 



ICNET tasking Itr of 15 May 1981. 
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• Periodic MIISA reports dealing with system component failures, 
frequency and type of transactions, non-CMI user requirements, and 
processing time provided the data from which conclusions were 
drawn about system capability and availability* 

• Information on system hardware/software configuration was obtained 
from extensive interviews with MIISA system managers and support 
personnel including representatives from the Honeywell 
Corporation. 

• Additional data were collected from interviews with school 
personnel concerning the handling of students during computer 
downtime and problems the schools were experiencing with the 
central CMI system. 

Analyses of the above data provided the basis for the conclusions and 
recommendations of this study. 

ORGANIZATION OF THIS REPORT 

In addition to this introduction, this report contains five additional 
sections and one appendix. Section II reviews and summarizes the CMI per- 
formance data from March to October 1981. Section III discusses the 
hardware and software configuration and limitations of the present CMI 
system. Section IV discusses the requirements and problems associated with 
non-CMI users being served by the CMI system. Section V includes an 
analysis of the relationship between computer downtime and training time. 
Section VI presents the summary and recormiendations. The appendix contains 
detailed information on various performance statistics. 
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SECTION II 

ANALYSIS OF CMI PERFOfWANCE 
(MARCH THROUGH OCTOBER 1981) 

Beginning in March 1981 performance statistics were collected for each 
shift at the CMI locations (Memphis, San Diego, Great Lakes, and Orlando). 
* Each week all four sites forwarded a CMI Computer Terminal Users Report to 

CNTECHTRA, MIISA, and CNET. Variables for which data were collected 
included CMI response time, number of interruptions, number of minutes the 
system was down during the shift, and the percentage of time the CMI system 
was available to the students. This section of the report summarizes the 
performance data for the CMI system between March and October 1981. The 
data for each variable are summarized for each of the four sites, seven 
courses, and two shifts. An observation consists of a measure of a variable 
taken for a given day, shift, course, and location. 

RESPONSE TIME 

The response time is measured (in seconds) from the time'a test answer 
sheet is ejected from the OPSCAN reader until the first character of print 
appears on the corresponding learning guide at the terminet. Response time 
is sampled for the first 10 minutes of each hour, beginning with the second 
hour and ending with the sixth hour of each shift. The response time does 
not include the time required to print the learning guide which varies 
depending upon the length of the guide. The procedure for sampling response 
data was specified by CNTECHTRA. 

Figure 1 shows the monthly distribution of response times for all 
observations between March and October 1981. The average response time has 
steadily decreased since the second quarter. By October j response time was 
averaging less than three seconds per transaction. The average response 
time at Great Lakes is slightly higher (approximately 4 seconds) and the 
remaining sites are averaging 2 seconds or less (figure 2). Approximately 
62 percent of the measured response times were 3 seconds or less. (See 
figure A-1 in the appendix.) 

The response times by day of week were considerably higher for Mondays 
during the March to June period. Since then, differences among the days of 
week have diminished and by October no significant differences existed 
(figure 3). Response times by day of week and month are illustrated in 
appendix figure A-2. 

Fewer than two percent of the observations had response times exceeding 
25 seconds, and fewer than 15. percent of the observations had times exceed- 
ing 10 seconds. Most of the longer response times occurred during the early 
part of the study period (figure 4). 

i 

The response time for the Propulsion Engineering (PE) CMI course at 
Great Lakes was slightly slower than for the remaining six courses which are 
on the CMI system (figure 5). This slower response time can be attributed 
to differences in course curriculum. The higher response time for the PE 
course causes the mean response time at Great Lakes to be slightly higher 
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Figure 2. Mean CMI Response Times by Location 
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Figure 4. Histogram of Mean Response Times 
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than the other three locations. There were no significant differences in 
response times by day of week or shift (appendjx figure A-3). 

Response time is, at present, very fast but it is evident that future 
system expansion will eventually lead to degradation of system response 
time. Because of the complex interrelationships of the system involving 
t me of day. hardware and software limitations, non-CMI user requirements, 
aJf^le maintenance and access procedures, it is difficult to ascer ain, 
with any degree of confidence, where the system will degrade. An actual 
scenario or simulation based scenario must be analyzed to determine where 
"bottlenecks" occur and to identify factors which contribute to slow 
response time. 

INTERRUPTIONS AND DOWNTIME 

The number of interruptions for each month between March and October^ 
are shown in figure 6. The interruptions are sutmied for each location and 
course. When the CPU is down at Memphis there will be an interruption at 
each course and location. The number of interruptions by day of week did 
not differ significantly. (See figure A-4 in the appendix.) The number of 
interruptions by site is illustrated in figure 7. 

The downtime for each interruption was not recorded so it was not 
possible to construct a precise distribution of mean downtimes. The 
information which was available was the total downtime and number of inter- 
J!lp?iSns fSr each day. The mean downtimes used in this study were estimated 
bv dividing the total downtime for each day by the number of interruptions 
in that day. The mean downtime during the study period showed a general 
downward trend. By the end of October, the mean downtime was «ti mated to 
be between 12 and 18 minutes per occurrence (figure 8). This represent? 
considerable improvement from the high in March which exceeded 30 minutes. 
The mean downtimes by location are shown in figure 9. Figure 10 is a 
histogram of the downtimes and again demonstrates that most downtimes were 
approximately 10 minutes or less. Figure A-5 in the appendix shows the dis- 
tribution of downtime duration for day of week. No significant differences 
were observed among the locations or for the day of week. 

The response time, number of interruptions, and average duration of 
downtime all interact to determine the total time the CMI system is unavail- 
able to the student. The reported downtime is intended to measure or track 
the entire time during each shift the system was not available for use. 
However, the method used for recording the downtime depends upon the staff 
or students placing a demand on the system, both when the system goes down 
and when it resumes operation. If there are no demands for service on the 
system when it goes down, then the actual downtime may occur at some point 
prior to the point at which the observation was taken. Therefore, under- 
certain conditions the actual observed downtime could be shorter than the 
actual time the system was down. Similarly, if the system resumed operation 
and there was no demand for service then the system could have been back in 
operation before the observation was taken. This would tend to lengthen the 
reported downtime. Deficiencies in reported data should, however, not be 
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serious and the following data on total downtime is submitted recognizing 
the above potential difficulty. 

The total downtime shown for each month in figure 11 is the total 
amount of time that CMI services were not available for all courses during 
the month* Downtime could occur at a site even though the central CPU was 
operating. If the CPU at Memphis were to fail, curtailing service to all 
courses and sites, then the total downtime as illustrated in figure 11 would 
be determined by adding the observed downtime for each course at each site. 
For example, two courses at each of two sites will result in total downtime 
of 240 minutes when the CPU goes down for 60 minutes. There was a signifi- 
cant downward trend in the number of minutes the system was unavailable from 
March to October 1981. During October the cumulative downtime for all 
courses at all sites was less than 2,000 minutes out of a total of approxi- 
mately 100,000 minutes for the month. The Basic Electricity and Electronics 
(BE&E) course had the highest number of minutes of nonavailability because 
BE&E is taught at all four CMI locations. The total downtime by day of week 
and course is shown in figures A-6 and A-7 in the appendix. 

SYSTEM AVAILABILITY 

The total downtime, measured at each location, can result from any 
number of causes. Although the entire system will be down when the CPU at 
Memphis is down, there are other failures which result in specific locations 
being down. Figure A-8 in the appendix shows the downtime for each 
location. Assuming that the minimum downtime in each month at the four 
locations represents the maximum amount that the CPU at Memphis could have 
been down, it is apparent that failures which occur at Memphis and cause the 
whole system to go down have been relatively low during the latter part of 
the study period. Actually, the failure rate of the CPU will be signifi- 
cantly lower than the above minimum times since many of the failures as 
observed at the site could be attributed to site problems. Failure of the 
central computer was not a significant problem during the study period. 

The percent of nonavailablity is computed by subtracting the nuirter of 
minutes the system is down during a shift from the total minutes available 
in the shift during the day and dividing the results by the total minutes 
available in the shift. 3 

Figure 12 shows the percent of time that the system was not available, 
averaged for all sites, during the period March to October 1981. Since 
March, monthly downtime has averaged less than 5 percent. Total system 
availability was very high during the period, and there was no evidence that 
system downtime was a problem. System availability by location and day of 
week at each location is presented in figures A-9 and A-10 in the appendix. 
System availability appears to be slightly higher at Memphis apd Great Lakes 
than at Orlando and San Diego although the differences are not great. There 
are no significant differences in availability among the days of the week. 



3 



ERIC 



CNTECHTRA msg 111848Z Mar 1981. 

14 



Technical Report 119 



300 



270 



240 



210 



180 



150 



120 



90 



60 



30 







WW www 
















***** 
















LJ a* a* 

1T**WW 
















***** 














***** 


***** 














***** 


***** 




mm mm mm mm mm 

WWW WW 










***** 


***** 




***** 










***** 


***** 




***** 










***** 


***** 


***** 


***** 










***** 


***** 


***** 


***ww 










***** 


***** 


ma ma ma a* a* 
***WW 


WW** w 










***** 


***** 


*** ** 


***** 










***** 


***** 


***** 


***** 










***** 


***** 


***** 


***** 










***** 


***** 


mm u mm ^ ^ 

***ww 


ww**w 










***** 


***** 


***** 


WW** w 








4f4f4f4f4f 


***** 


***** 


mm u u u u 

***ww 


www WW 










***** 


***** 


***** 


***** 








4f4f4f4f4f 


***** 


***** 


***** 


***** 








ff4f4f4f4f 


***** 


***** 


***** 


***** 








4f4f4f4f4f 


***** 


***** 


***** 


***** 








##### 


***** 


***** 


***** 


***** 




***** 




##### 


***** 


***** 


***** 


***** 




***** 




4f4f4f4f4f 


***** 


***** 


***** 


***** 




****** 




4f4f4f4f4f 


***** 


***** 


***** 


***** 




***** 




4f4f4f4f4f 


***** 


***** 


***** 


***** 




***** 




4f4Hf4f4f 


***** 


***** 


***** 


***** 




***** 




4f4f4f4f4f 


***** 


***** 


***** 


***** 




***** 






***** 


***** 


***** 


*it*** 




***** 


***** 




*#*** 


***** 


***** 


***** 


***** 


***** 


***** 




***** 


*«»** 


***** 


***** 


***** 


***** 


****<« 




***** 


***** 


***** 


***** 


***** 


***** 


***** 




***** 


***** 


***** 


*)f*** 


***** 


***** 


***** 




***** 


***** 


***** 


***** 


***** 


***** 


***** 




***** 


***** 


***** 


***** 


***** 


***** 


***** 




***** 


****4t 


***** 


***** 


***** 


***** 


***** 




*4t**'i« 


***** 


***** 


***** 


***** 


***** 


***** 




***H* 


*-k^H** 


***** 


*»*** 


***** 


***** 


***** 




***** 


***** 


*i^*** 


***** 


***** 


**H*H 


***** 




***** 


***** 


***** 


***** 


***** 


***** 


***** 


4f4f4f4fff 


***** 


***** 


***** 


***** 


***** 


***** 


***** 


4f4f4f4f4f 


***** 


***** 


***** 


***** 


***** 


***** 


***** 


MAR 


APR 


MAY 


JUN 


JUL 


AUG 


SEP 


OCT 



MONTH 



Figure 6* Total Interruptions for All Courses 




Technical Report 119 



100 



• •••• 

• •••• 

• •••• 



••••• 

• •••• 
••••• 

• •••• 
»•••• 
••••• 

• •••• 

• •••• 

• •«•• 

• •••• 

• •• 
»•••• 



• •••» 

• ••• » 

• •••» 

• •••• 

• •••• 

• •••» 

HAH 



120 ♦ 



100 



DC 

a 60 











• ••»• 








• •••• 




• •••• 


• •••• 






• •••• 






• •••• 


• ••• • 






• • • 








• •••• 






• »•• • 












• •••• 






»•••» 




• »•• • 




• •• 






• •••• 




• »••• 




• •••• 


APR 


MAY 




JUL 


AUG 




OCT. 



>10N1H 
MEMPHIS 







••••• 


• •••• 
















• •••» 

• •••• 






»•»•• 




•»•••• 


• •••• 




• •«• • 










• •••• 






• • 














• •••» 








• •••• 




»•••• 




• ••• • 








• «••• 


••••• 


• •••• 


• »••• 




• •••• 


»•••• 




»•••• 


• •••• 






• •••» 








»»• •» 




• •••• 




• •» 


• »••• 




• •••• 


»«••• 




• •••• 






• •••• 


» •••• 


»•••• 


»•••• 




• 


• •••• 




• •• 




• •••• 


• •••• 


• ••»• 


• •••• 


• •••• 




• •• •• 




• •••• 




• ••»• 


• •••• 






• •»»• 


»•••• 


»•»•« 


»«»«<i 


• •••• 




















APR 


MAV 


JUN 


JUL 


AUG 


SEP 


"oct" 



PONIK 
SAN DIEGO 



I7Q 



60 















• •• 

• •••• 

• •• 

• •«•• 

• •• 


••••• 


• •••• 
»•••• 

• ••• • 

• •••• 

• ••»• 


• •• 

• •••• 

• • ••• 






• 


• 


• •••• 


• •••• 


• •••• 




• •• 

• •••• 


••••• 


• ••• • 

• •••• 


• •••• 

• •••• 


<••»•» 


• ••»• 


• •••• 

• • ••• 

Acn 


••••• 

MAY 


• ••»• 
• 

JUN 


• •••• 

• » 

JUL 


AUG 


• •••• 



Jl 60 



20 





••• 


• • «•• 














• • ••• 


, •»••• 




••••• 












• •• 














****** „ 






• •••• 




••••• 






• •••• 


• ««•• 










•»•• 




• 


• •••• 


• •••• 


• •••• 






•••• 




• •• • • 






• •••• 




• 


•••• 




• •••• 


• »••• 




••••• 




(••••» 




• «•• 


• •••• 


• • ••• 




•*•••• 




• •••• 


••fl* 


• ••• 


• •••• 


»•••• 




• •••• 


• •••• 


• •••• 


•••• 


• ••• 


• •• 










• •••• 


• 


















HAM 


APM 


'mav 


*'jUN ' 


'**JUL*' 


AUG 


ser 


0C1 



HONIH 
GREAT XAKBS 



H0S1H 
ORLANDO 



Figure 7. Total Interruptions by Location 



ERIC 



16 



is 



Technical Report 119 




Figure 8. Mean Downtimes by Month for All Observations 
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SECTION III 

COHPUTER MANAGED INSTRUCTION HARDWARE/SOFTWARE CONFIGURATION 

The CMI system hardware configuration is functionally depicted in 
fiaure 13 As the figure shows, there are many subsystem elements involved 
il^the processing of one compleie CMI transaction. This system charactens- 
c a oSrcoul J theoretical 1? lead to increased downtimes gn^ re^PJJI^^^^JI ' 
However this does not appear to be the case as evidenced by the statistical 
dau Find ngs indicate that excessive downtime and degraded rfponse time 
a?e not ser Sfs problems for current student loading levels. It is not 
knowrhoSeJer! how many additional students and courses can be added before 
uSac^^ptaSle levels of downtime and response time will be experienced. The 
e emen of tM^Hssue which relate to, computer hardware and software con- 
figuration/capabilities are addressed in this section of the report. 

EQUIPMENT HARDWARE CAPABILITIES/LIMITATIONS 

Each of the subsystem components shown in figure 13 P^^f 9';'"s / ^""ction 
within the system and each is subject to failure. Consequently, a ^^ailu'^f 
a 0 ated wu5 Sata entry, data communications T^^^ Plexing or proce g 
will temporarily cause partial or total system failure. The partial failure 
c e 1 gSrSe limited to an OPSCAN or terminet ^^iJ^HJ^fS fed"1h?s Le 
nnp of aooroximately 150 input-output clusters would be affected. This type 
S^failSfrw^ ld normally be corrected in 10 to 15 minutes by replacing the . 
filled u^it with a spare; Then the failed unit would be repaired to main- 
tain a backuS caoability A total site failure could be caused by an inter- 
JSJtiSn ?n «?ca on which results in network separation between the 
s te ^SnJentjror (Honeywell Level 6) and the central processors Honeywell 
series 60 dual processor) at Memphis. If other operable corrmunicat ons 
ciJcu ts cou?d not be used, this failure would remain for the duration of 
tJe communications li.e failure. However, i^f^^d ^^^^ J^^^^ ^P^f^jj?"' 
at the site losing communications. Additionally, the probability ot this 
occ rHng is reasonably low. The most serious failure ^^^Jd be one in which 
conmon elements affecting the dual central processors would fa 1. Jhis 

hp Hmp to Dower failure fire, flood, or some other similar serious 
S uJre ce For'aaslf Jhis nature, conceivably the CMI system might be 
S fo? d^vs or weeks. Although this type of, failure is not very Jike-ly.: 
aHccS^rence would impact a significant portiin of tota Navy training. It 
?s ?o? rSotentia ity such as this that a CMI manual backup system should 
a wajs be'^availab e. For networks utilizing a single centra processing 
fur^igh reliability and reasonable redundancy (both of which have been 
designed into the existing CMI system) can only decrease the ^sk of total 
sfstlm failure. Only distributed autonomous transaction processing will 
assume that total CMI system failure (all subsystems inoperable concurrently) 
will not occur. 

The following discussion will be directed toward specific network sub- 
system elements in order to provide a more detailed assessment of system 
capabili ties/1 imi tati ons . 
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Figure 13, CMI System Hardware Configuration 
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IHPlfT-OllTPUT LEARNING CENTER CLUSTERS. The input-output learning center 
clusters can be used to input student tests, each having up to 50 multiple 
choice respSnsL. through an OPSCAN-17 optical mark reader or to output learn- 
ina quidef S^a Terminet 1200- keyboard/print terminal. This input-output 
JS?nSe can al 0 be sed to ente? administrative transactions and to rece ve 
^Hm^n?ctrativP resDonses Input-output control logic for these clusters is 
?oSu S iJ ?heTse o^the OPSC^^^ Snit. A communications interface to the 
SoSeJiSll Level 6 concentrator is established via GDC-202-9B modems. 

Because the OPSCAN reader and the terminet printer are electro-mechanical, 
failure ?a?ls for these devices are higher than cil^^^c .rp 

elements These units do have a relatively high failure rate but spares are 
no?ma y'kept on Sand, if available, to replace failed units. By using this 
maintenance'strategy. cluster failures at ^iteswi backup units do not 
obiectionably degrade system capability or availability. It does appear. 
SoSr h t locations without'a backup capability could seriously degra e 
system availability. The control logic and modems have proved to be highly 
reliable and the cormiuni cat ions lines which connect the clusters to the 
concentrator are similarly reliable. This input-output configuration. 
altSoSgh n°ot The besi or most reliable by today's ^tandar s. provi es an 
adeauate svstem capability. Many improvements are possible for this network 
dT h fs han"Se?J device injut. 'higher speed input. J f ^^l^f^^f p^^^^^^^^^ 
outDut and keyboard/display testing terminals. However, the present configura- 
t oS prS; defsatisfactSry^erformance and any proposed improvements would have 
to be individually assessed on the basis of cost-benefit projections. 

HONEYWELL LEVEL 6 CONCENTRATORS. The Honeywell Level 6 concentrators are in 
essence communications computers which multiplex inputs from the learning 
center clusters for transmission to the central computer complex in Memphis. 
mTn 5ata in the form of learning guides or administrative responses are 
also routed to the proper receiving station via this subsystem. Since the 
Slta rae on outgoing 5r incoming communication li"" ^s re a ively low in 
terms of computer capability, response time should not be limited. In addi- 
tion ?he ?eliability of this equipment has proven to meet or exceed expecta- 
t SSs Although other processing, beyond that required for comnun.l cat ions. 
Uaccompi shed within this computer, only a failure affect ng multiplexing 
or communications would impact student testing. Because a 1 sites do not 
have subsystem redundancy, it is possible that failure ""Jd cause CMIinter- 
ruDtions for extended periods many hours . Failure data collected for the 
5a?t 6 mon?hs. however, do not show this to be a prob em and. in the opinion 
of experienced personnel, continued high reliability is expected. 

FRONT-END COMHUNICATIONS PROCESSORS. The network arrangement between the 
student input-output clusters and the concentrator is repeated in concept at 
the Memphis^ host computer site. For this application a front-end communi- 
cations processor is utilized to multiplex inputs from many site locations. 
This 5?oCe?so? aJts as a temporary buffer and switch directing incoming 
transactions to buffer locations in main processor memory. This provides 
temSo?ary transaction data/storage while awaiting central processor service 
When service is completed, the transaction response information is routed to 
thfproper output coirmuniJations channel via the communications processor. 
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Discussions with system personnel have indicated that the memory limitation 
on the front-end processors sometimes results 'in system overload and tem- 
porary failure. These failures are not normally catastrophic since a system 
reboot will return the system to operational status in a period of minutes. 
It is a problem, however, which should be diagnosed further to determine 
what corrective measures should be considered. 

HONEYWELL SERIES 60 DUAL PROCESSORS AND PERIPHERALS. The equipment at the 
Memphis central CMI processing center consists of 2 Honeywell series 60 
processors, 23 lOO-MB disk drives, 11 magnetic tape units, 4 front-end 
communications processors, 2 1200 LPM printers, a card reader, and a card 
punch. The processors share one mega word of memory and are configured to 
service multiple users. The multi-tasking, multi-processing onerating 
system combined with the data b.ase management and input-output handling 
systems combine to provide a powerful and effective computational complex. 
Although the CMI system uses only .a portion of the total resources available 
(for example, CMI uses only three of the 23 disk drives available), it is 
the highest priority user and is not affected by other users in terms of 
response time. A system crash could result from defective non-CMI applica- 
tions software or front-end processor overload. However, this possibility 
is not considered serious because most recorded failures in this category 
have been corrected within reasonable time limits. The peripherals have 
also proven to be reliable and of sufficient capacity to handle peak 
loading. An analysis of system capabilities has shown that there are some 
CMI software characteristics which would limit system expansion. However, 
it appears that these characteristic limitations could be corrected with 
some system redesign. This topic is discussed further in the following 
paragraph. 

SOFTWARE CAPABILITIES AND LIMITATIONS 

The system resident software in the CMI network consists of operating 
systems, data base manaaement systems, communications programs, and utility 
programs. This software, with a few minor exceptions, has proven to be 
reliable. Some relatively minor modifications have been made to the 
resident vendor developed software for special applications. 

The CMI software which controls transaction processing is considered to 
be of primary importance. The software consists primarily of an evaluation 
program, which provides the test evaluation and learning guide generation 
capability, and a number of administrative transaction processing programs. 
Although the administrative programs such as registration, class rosters, 
student progress, and those concerning student flow are necessary for 
effective school management, normally they are not competing for time with 
evaluation. They are considered administrative batch operations and are 
normally scheduled for minimum impact on student testing. 

The evaluation program which controls the analysis of, and response to, 
student tests is the primary response time controlling program. At present, 
' the maximum test transaction throughput rate is* approximately two trans- 
~ actions per second (120 per minute or 7,200 per hour). If it were possible 
to maintain this rate for two six-hour shifts, which is unlikely, 86,400 
test transactions could be serviced. This assumes that no administrative 
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transactions are competing for time and that all answer sheets are properly 
cSded t also assumes a continuous flow of test input. For the nieasured 
case, approximately 35,000 test transactions are serviced each day with the 
balance of the time being utilized for administrative transactions and error 

'tJlSsactfons witMeriodi of nonuti lization in^5^^P|S^«d,J^;°"f3;f,,^ Jal 
For the current student loading of approximately 8,500, the existing evaiua 
SSn capaSnity is adequate. Making Changes to the evaluation program to 
allow for multi-tasking operation within the evaluation program would have 
the effect of handling many test transactions at a time as opposed to one at 
a time for the present case. 

Upgrading the evaluation program appears to be a relatively simple 
solution to satisfy a potential need for greatly increasing throughput. 
However, for a change of this nature, careful study should precede develop- 
ment. This may not be a satisfactory solution unless significant disk and 
file restructuring accompany the multi-tasking approach. If a single aiSK 
access to a course file locks that file out for a following transaction, a 
wait period of 20 to 50 milliseconds or more might result which could 
partially negate the expected benefit of multi -tasking. The net result of 
this situation might be a moderate improvement in throughput and no* ^Je 
significant improvement expected. File reorganization and new approaches 
to course file development, if effectively accomplished, together with 
m2lti-talking, should minimize the number of disk accesses and improve the 
overall response time. By following this approach, high Pnonty file 
handling could be limited to no more than two disk accesses as compared to 
the present 5 to 12. This would then be followed by housekeeping trans- 
actions which would be accomplished in background mode. 

Even if these improvements in the evaluation were accomplished, other 
hardware and software response time constraints might occur. These con- 
straints might be due to data net overload, student cluster overload, or 
inadeSuate buffer storage. It is suggested that a total system^network 
ISalysis be done to determine the maximum throughput at each node before any 
single measure is taken to improve system performance. One obvious approach 
to iystem expansion, if required, would be to provide a distributed process- 
ing capability at each site. Although this approach has certain detrimental 
effects which would offset some of the benefits to be gained, it appears to, 
be a reasonable expansion option. This issue is discussed in the following 
paragraph. 

DISTRIBUTED PROCESSING CONSIDERATIONS 

The phrase "distributed processing" is often suggested as a solution to 
many of today's data processing problems. Whether the problem is response 
time insufficient storage capacity, or data base management inadequacy, 
there is a tendency to favor a corrective measure in the form of distributed 
processing. This assessment of distributed processing stresses the need for 
cautious evolutionary development when considering this alternative for CMi 
system expansion. 

It appears inevitable that distributed processing will play a signifi- 
cant role in the future and coOld provide the means for greatly increasing 
current system capacity. However, it should not be considered a potential 
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cure all. The primary question which should be addressed when assessing 
this, option is, how many and which processing functions can be efficiently 
distributed. There are also questions concerning loose or tight coupling 
within the network, the necessity for distributed data base management, com- 
munications protocol, privacy/security/integrity, and network • 
management/control. Dispersing many CMI processing, storage, and reporting 
functions without maintaining strong and effective centralized policy 
development and management control could bring about a degradation in CMI 
system performance instead of the desired improvement. 

The distributed approach has many advantages. It would not be 
necessary to cbmiiiunicate student test response data hundreds or thousands of 
miles for response analysis and learning guide generation as is now the 
case. Test transaction processing is well within the capability of medium 
scale computers which could be located at remote sites. A remote computer 
of this type could also provide the processing functions associated with 
class roster generation, predicted completion time (PCT) resource 
allocation/scheduling, and site level administrative support. However, it 
might not be in the best interest of the Naval Education and Training 
Command to distribute such functions as student registration, student record 
keeping, student tracking, and training pipeline management. These examples 
are not offered as reconmendations. They only demonstrate the extent of the 
analysis required before making hard decisions relating to CMI configuration 
changes. They also demonstrate the necessity for an in-depth analysis of 
CMI long-range requirements before selecting a course of action for system 
redesign. 
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SECTION IV 
NON-CHI USERS 

The Honeywell 6000 computer system, located at Naval Air Technical 
Training Center (NATTC), Millington. is used for '^'^e than CMI support (s^ 
figures 14 and 15). This system supports numerous other functions including 
naval technical training, recruit training, and miscellaneous activities. 

The largest single user (where use is measured by processing time) is 
the Military Personnel Information System (MILPERSIS). Information is 
passed to MILPERSIS throughout the day to update various data concerning 
students within CNTECHTRA. Intense MILPERSIS use of the Honeywell 6000 
system is reserved for the 1800-2300 Central Standard Time period when CMI 
use is very low. 

The second and third largest users of processing time are CMI and the 
maintenance of the MIISA General Computer Operating System (GCOS). 
Together, these three large users-MILPERSIS CMI. and MIISA operating 
systems maintenance-account for approximately 85 percent of the Honeywell 
6000 processing time. The remaining 15 percent is used for numerous 
functions.' including primarily: 

• Standard Transfer Directive Module (STDM) for ordering the trans- 
fer of students 

• NATTC, Millington, civilian and military payrolls, and civilian 
personnel support 

• NATTC, Millington. logistical functions; e.g.. Navy Stock Fund and 
Resource Management System 

• U.S. Army Corps of Engineers support 

• Individual Flight Activity Report Subsystem (IFARS) for managing 
flight personnel. The Honeywell 6000 relays IFARS data from 
Memphis to Pensacola for central processing. 

• Surface Warfare Officer School support 

, Availability Reporting and Tracking Module (ARTM) and Recruit 
Accession Module (RAM) to aid in personnel management within the 
Recruit and Student Training Commands. Although this support is 
provided primarily on Level 6 systems, the Honeywell 6000 does 
provide some central processing support. 

• Naval Air Maintenance Training Group and Air Maintenance Detach- 
ment support 

• Computer Driven Training System which provides CAI-like instruc- 
tion in computer use. 



O 29 3. 

ERIC 




MONTH-1981 

33 

32 Figure 14. Distribution of Honeywell 6000 Processing Time (%) Among Users 



ERIC 




34 



MONTH-1981 

Figure 15. Distribution of Honeywell 6000 Processing Time (Hrs) Among Users 



Technical Report 119 



Currently, there -is no statistical basis for relating degraded CMI 
performance to non-CMI user applications. Performance indicators reviewed 
show typical CMI response times of 3 to 5 seconds and overall system 
availability exceeding 95 percent. In addition, there is no indication that 
non-CMI users cause more than a proportionate number of failures of those 
recorded. It is obvious that non-CMI use will increase the probability of 
total system failure, but it can not be determined at the present time if 
this increased risk of failure would warrant a reduction in service to non- 
CMI users. By allowing multiple users, system utilization is increased and 
the return on computer investment is positively affected. Although there 
are a number of ways to improve the operational availability of the CMI 
system, as discussed in section III of this report, it appears ^doubtful that 
limiting non-CMI use beyond current levels would have any significant 
impact. 
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SECTION V 

RELATIONSHIP BETWEEN CMI DOWNTIME 
AND STUDENT TRAINING TIME 

IMPACT OF COHPlfTER DOWNTIME 

The most obvious and potential impact of CMI downtime on the cost of 
training arises from extending the time required for students to complete 
training. An objective determination of how CMI downtime affects training 
time requires the collection of data on student activity during downtime 
and the use of empirical measures of training time as a function of CMl 
availability. Such data were not collected during past CMI downtimes and 
during the period covered by this, study availability was so high that there 
was an insufficient number of adverse effects from which to deduce a func- 
tional relationship between downtime and training time. 

At least two important factors contribute to the relationship between 
training time and CMI downtime. The first arises from the role of CMI in 
the instructional process. The Navy CMI system itself is not designed to 
provide instruction during the time the student interacts with the computer. 
Apart from the incidental learning which takes place during tests, the 
majority of instruction occurs while the student is in the carrel and not 
interacting with the computer. The computer provides periodic Performance 
evaluations and directs the student in future study assignments by issuing 
learning guides. Consequently, when the computer is down those students who 
are not ready for a performance evaluation will not be directly affected. 
If the downtime interval is very short (as most have been in the last four 
months) and the frequency of student interaction is low, then very few stu- 
dents would even be aware that the CMI system was down and there would be no 
impact on those students. For those students who were affected, the average 
iaUing time ?or the computer would be only a fraction of the computer down- 
time, assuming students demand service at a constant and uniform rate during 
the shift. 

The second factor which impacts on the relationship between training 
time and CMI downtime is how the student is managed in the classroom once 
the student demands service and finds the computer down. Arguments are fre- 
quently advanced that the amount of lost training time for any student can 
be measured from the time the student demands service finds it unavail- 
able to the point he/she obtains the requested service. This argument must 
be predicated on. the assumption that learning stops when required CMI 
service is not available. Since the computer plays its role in the manage- 
ment of instruction, and not the instruction itself, such an assumption is 
untenable. 

One alternative for obtaining data from which to derive the relation- 
ship between training time and computer downtime would be to devise an 
experiment in which data on student queues, training time, and effectiveness 
would be compiled and analyzed following controlled CMI shutdowns. Such an 
experiment was considered not to be feasible with the operational CMI 
system. 
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An alternative to the experimental approach is to perform a logical 
analysis of the problem using qualitative data drawn from previous research 
dealing with CMI systems and discussions' with school management as to how 
current CMI shutdowns are handled. The following discussion is based upon 
an analysis of those factors which determine the training time required by 
students, an analysis of how the computer/CMI system interacts with those 
factors which determine training time, and an assessment of the management 
of students at the schools when the CMI system is down. 

FACTORS INFLUENCING STUDENT TRAINING TIME 

A review of training literature shows that "student training time" has 
many components. Carroll (1963) identifies five factors whi civ interact to 
determine the total student training time. These include: (1) time allowed 
for learning, (2) time the learner is willing to spend, (3) time required 
because of th2 student's ability, (4) ability to understand instruction, and 
(5) quality of instruction. Bloom (1976) has demonstrated that through 
individualizing instruction, considering each student's unique status with 
respect to the above five factors, certain conditions of learning can be 
established which facilitate the student in learning. The importance of 
these conditions for the present problem is evident in the statement, '... 
what any person in the world can learn almost all persons can learn if 
provided with appropriate prior and current conditions of learning" (Bloom, 
1976, p. 7). When these conditions of learning are optimum, we can get 
almost anyone to learn almost anything. There are six conditions: 

1. Prerequisites (PRQ). These are the knowledge and attitudes that 
students bring to the learning situation based on previous experience. 
ASVAB scores and reading and computational scores are often used to assess 
this condition. Attitudes are reflected in measures of motivation and 
perseverance. The best adaptive instruction accommodates student variation 
among the prerequisites. 

2. Cues (CUE). This condition involves the ways the instructor 
informs stuHents what they are to do, and includes learning objectives, 
verbal explanations, demonstrations, and models. 

3. Participation (PAR). In order to learn, the student must overtly 
or covertly do something. Both instructors and the instructional materials 
must keep the student's mind intently engaged with the subject matter. A 
very high relationship exists between such mind engagement and amount 
learned. Such "mind engagement" time is not necessarily related to the 
amount of time a student spends in the carrel. PAR is the study time that 
remains after subtracting wasted time from the time a student spends in a 
learning center or laboratory. 

4. Reinforcement (RNF). A reinforcer is part of the reward or 
motivational system that strengthens the behavior that pVecedes its 
administration. The definition is circular in that if the behavior is not 
strengthened, whatever was administered was not a reinforcer. In the 
science of instruction, here is where the "art of teaching" comes in. There 
can be no formula for the use of reinforcers. It takes a wise and sensitive 
instructor to'know when to use external reinforcers such as recognition or 
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privilege and internal reinforcers such as leaving the student alone when 
the instructional materials are obviously strengthening PAR. 

5. F eedback (FBK). This is the information that informs students of 
the degree 'to which their practice is discrepant from that which they are 
supposed to be practicing. Sometimes FBK and RNF are the same; other times 
FBK is neutral. Tests, critiques, and oral examinations are used for this 
function in an individualized instructional system. 

6. Cor rectives ' (COR)'. After the FBK shows a discrepancy between the 
raquired and the demonstrated response, the corrective prescribes some sort 
of learner activity that will eliminate the discrepancy. 

Wherever there is efficient and effective instruction, individualized 
or lock-step, these six conditions of learning are present. 

A L06ICW. ANALYSIS OF QUEUES AND INTERRUPTIONS IN THE NAVY CMI SYSTEM 

The Navy CMI system is designed to assist primarily in two, of the above 
six conditions, namely feedback and corrective functions. The CMI program 
is tailored to assist the learning center instructor (LCI) in providing 
alternative forms of tests, retakes of examinations, and P'^f criptions for 
corrective actions for student weaknesses. Information contained in the LMi 
system Student Progress Reports also serves the prerequisite and partici- 
pation functions. If a student progresses through one or two instructional 
modules every day, takes 30 minutes to take a test and score it at the 
OPSCAN terminal, and demonstrates mastery on about the second attempt, a 
conservative estimate of the proportion of his or her total learning center 
time spent in test taking would average approximately one hour per day. 
Even during this period, it is estimated that the student would interact 
with the computer for less than five minutes of the time. When the CMI 
system is down, learning need not stop, although the sequence in which the 
student undertakes the learning experience is usually modified. 

The flow diagram in figure 17 is based on how a CMI learning center- 
activity might operate when the computer is down. There are five key points 
in the model where student queues might be expected to develop following an 
extended period of computer downtime. They are awaiting: 

1. the first lesson or module assignment in the course, 

2. instructor help or approval to attempt a formative or module 
examination, 

3. examination scoring and study assignment at the OPSCAN terminal, 

4. instructor diagnosis, counsel, and prescription followijig failure 
to demonstrate mastery on a test, and 

5. assignment or materials for the next lesson. 

These points are noted as QUEUEl through QUEUES, respectively, in the flow 
diagram. As the diagram shows, four of these possible delays are directly 
related to the LCI's availability of time. 
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Following a shift to the manual mode because of a computer failure, the 
LCI must become occupied with keeping the students productive. This involves 
keeping them actively pursuing course goals and manually entering updated 
student information into the records system. Perfectly designed instruc- 
tional materials would support the important conditions of learning so that 
the instructor would be free to carry the computer's share of the management. 
Since the instructional materials are seldom so designed, it is reasoned 
that the instructor would eventually become involved in all phases of the 
instruction and such manual management activity will begin competing with 
the students' needs for cues, feedback, reinforcement, and correctives. 
Many factors would determine how long the computer must be down before this 
overload would become a serious problem/ It is estimated that on the average 
it would not become a serious problem for downtimes under 1 hour. 

MAMAGEMENT OF STUDENTS DURING COMPUTER DOWNTIME 

Figure 17 illustrates how instructors might intervene at each QUEUE 
position should the computer system fail. Presently, instructors do not 
usually intervene (at QUEUE2) to determine, by oral examination, that the 
student is ready for the test and has a high probability of being successful 
—but if they were required to go into a manual mode this intervention would 
be necessary. Presently, the QUEUES requires a short wait for scoring and 
- test results— but in a manual mode, this wait would certainly be longer. 
Instructors do not necessarily override the study assignment accompanying 
the printout of test results in order to give a personal diagnosis of learn- 
ing difficulties— but they can, and in a manual mode they would provide the 
personal diagnosis. 

A hypothetical example of how a competent LCI who knows his or her 
students should manage computer downtime would be as follows: 

Students who are known to have conceptual difficulties with the pre- 
requisite learning modules and are running far behind their predicted com- 
pletion time (PCT) are requested to spend additional time studying in their 
present module. Such study could be in an additional method of presenta- 
tion, such as the summary, narrative, or programmed instruction mentioned in 
NAVEDTRA UGA. It could be in the form of an elaboration of the module goals 
by film, filmstrip, trip to the lab, or peer instruction by an advanced stu- 
dent. The goal of such activity would be for the student to demonstrate 
mastery of the present module on the first attempt. Such "overlearning" is 
not necessarily inefficient for this student; it may be the learning-how-to- 
learn, or the confidence-builder, that will cause him or her to "takeoff 
after the CMI system comes back on line. 

For students who are ahead of their PCT and who generally demonstrate 
mastery of modules on the first test, the instructor would present subsequent 
modules. The time required for testing seems to slow these students' progress, 
They study in the learning center or lab for the knowledge, not just to pass 
tests. Such students can go back and successfully take several module tests 
without any debilitating effects long after the computer comes back on line. 

For the majority of students between these two extremes, a period of 
computer downtime will require an LCI who has the ability to differentiate 
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among students and manually make assignments. In order to ensure that the 
IZlnls make effective and efficient use of their he assignm n s 

must be based on the students' progress with respect to their PCT, their 
motivation, and their self-reliance. Instructors must be able to 
e?flctiv?i; utilize audio-visual aids and °ther materia s which are 
available to ensure student progress. The management of the student during 
coSJufer downtime provides an opportunity for LCIs Jo demons rate how 
essential they are in facilitating the conditions of learn ng. It is not 
lasrand periods of downtime are not without stress, but instructors should 
be selected who can handle problems which arise during periods when the 
computer temporarily fails. 

EVIDENCE OF THE IMPACT OF COMPUTER DOWNTIME ON TRAINING TIME 

There is little eifipirical data demonstrating the effect of computer 
downtime on training. The evidence which does exist seems to suggest that 
computer downtime need not halt student progress. 

Several studies (Oudd, McCombs, and Dobrovolny (1979); Diamond (1969); 
Anmentoi^p" Morris, Ind Miller (19731) have demonstrated, in general, that 
even though the CMI/CAI systems they were studying had interruptions, 
students could still maintain acceptable progress if their time were 
adequately managed. Difficulties which had to be overcome included the need 
to provide adequate student feedback and student time management, in,, 
general, the manual management methods were less efficient than the 
computer. 

Two Navy CMI sites (Memphis and Orlando) were visited to obtain data on 
the effect of computer downtime on training. The following questions were 
posed to school staff personnel who have had experience with the CMI system: 

1. How often does the computer system go down? 

2. About how long does the coinputer usually stay down? 

3. What do learning center people do when the computer is down? 
4 Do you have a manual or back-up system? 

5. How long are learning center queues per downtime? 

6. How long must the system be down to affect student time to 
mastery? 

Individuals who responded to the above questions were generally 
cognizant of problems which existed before March 1981-the last time there 
were any appreciable downtime problems. There are several generalizations 
to be drawn from responses to these six questions. First, there have been 
very few problems since March 1981 in terms of response times or inter- 
ruptions. When there were interruptions, they have not lasted "^o^^ than 
a few minutes. Second, most learning centers do not have a paper and pencil 
Lk'up system for testing, so during downtime students do addition a study. 
If the students are taking a test, they just review their work a little 
ionger before submitting it to the OPSCAN. If the downtime is long the LCI 
assigns the students the next instructional module in their learning 
?eqieSce F nally, queues following the return of the computer are usually 
SeSrshon, a 1 though there were examples of nearly 2-hour queues which 
occurred over a year ago. 
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The question of how long the system must be down before there would be 
an adverse effect on training time produced responses which varied from 15 
minutes to 1 hour. Much of the variation in these estimates could be 
attributed to how well the school was staffed, prepared, or had available 
back-up procedures. 

The cost of computer downtime can be computed under two alternative 
assumptions. The first assumption (Case I) would represent an upper limit 
to the cost of computer downtime, except perhaps for extremely long 
-downtimes of several days during which the entire training program might 
need to be restructured. Case I assumes that for each minute a student 
spends waiting for the computer that training time is extended on a one for 
one basis. The formula for computing lost training time for Case I is as 
follows: 

■ L = (1/2)(C)(S) 

where: L = Lost Training Hours 

C = Hours Computer was down 

S = Students demanding service per hour 

The second assumption (Case II) is based on assumptions about the 
effect of computer downtime on training time which seem more realistic as 
determined from data collected for this study. First it was assumed for 
downtimes of 30 minutes or less that there would be no measurable increase 
in.jtr:a.in;irig-_.tjme. For downtimes with a duration between 30 minutes and 60 
minutes, training time would be extended for one-half the student waiting 
time. And, finally, for downtimes which exceed 1 hour there would be an 
extension of training time equal to the amount of time all students spend 
waiting for the computer. The equations used for Case II are as follow: 



L = 0: If D < 1/2 Hour 

L = (1/2)(1/2)(C)(S): If 1/2 Hour < D < 1 Hour 

L = (1/2)(C)(S): If D >1 Hour 

where: D = Duration of Interruption. 
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SECTION VI 

SUMMARY. CONCLUSIONS. AND RECOMMENDATIONS 



SUMMARY 
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The present performance of the CMI system, as deduced from data 
collected during March to October 1981 showed that response times were 
satisfactory, averaging less than three seconds per transaction, and the 
frequency and duration of interruptions were relatively low. For most days, 
the system was available 95 percent or more of the time during which the 
schools were in operation. While there were rather severe problems in 
response time and interruptions prior to March, improvements in system 
software and operating procedures have significantly reduced these problems. 
The system does not have the most efficient design in terms of disk access 
and other system software and potential exists for upgrading system capacity 
by implementing some design change in these areas. 

The largest single use of the present CPU is MILPERSIS followed by CMI 
and then the maintenance of the MI ISA operating system. Together these 
three uses account for 85 percent of the total processing time which is 
used. However, all uses together do not fully utilize the available 
capacity although there are periods during the day in which the system is 
nearly fully utilized. Computer Managed Instruction is effectively given 
precedence over all other users through allocation of CPU time. 
Consequently, at the present level of utilization the non-CMI users do not 
appear to significantly impact CMI processing. Even during those periods in 
which the capacity is being fully utilized, CMI is allocated all the 
processing time required. There is some minor contention for disk access 
between MILPERSIS and CMI and under worst case conditions CMI performance 
degradation is estimated at 5 to 10 percent. It is expected, however, that 
continued expansion of CMI will ultimately force delays in the processing of 
non-CMI transactions. As previously indicated, the level of loading at 
which delays will become significant can only be determined by conjecture 
using currently available data. Development and maintenance work on both 
the CMI and non-CMI software and hardware is usually scheduled during off 
duty hours. Such scheduling reduces the impact of development work on the 
effective operation of the CMI system. 

CONCLUSIONS 

The essential and most relevant management problem is to determine 
alternatives which are both technically and economically efficient to enable 
the system to maintain performance and to provide for future growth. Before 
any meaningful economic analysis can be performed, it will be necessary to 
determine at what point increased CMI loading will cause problems. It will 
then be necessary to determine the source of the problems, and then to 
evaluate alternatives for overcoming them. 

Factors which would adversely impact the operation of the current CMI 
system could only be postulated and cannot be deduced or observed from any 
degradation in performance of the CMI system as it is presently operating. 
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File access procedures, hardware limitations, system design, non-CMI process- 
ing reSuirements, fluctuations in CMI processing requirements, and ma y 
other factors interact in such dynamic fashion that even those intimately 
?a^ilia? with he exist ng system cannot conclude at what level of increased 
ope nignf cant degradation in response time would occur While it 
is rpasonable to assume that the system will become overloaded at some 
ncreaseS leve 0? peration, it is not readily apparent what hardware or 
sSnware components I management problems will prove to be the "weak" link. 

Two methods can be used to determine what improvements "ill be required 
in order to maintain system performance as more students are placed under 
JKeCMr system First! a comprehensive simulation model would Provide one 
means of determining the system limitations f .^^^^ ^^^a^ b u e to 
if" type relating to system upgrading. Simulation "uld also be used to 
determine if, and to what extent, system Performance wii be da raded by 
Hpmands for Service from non-CMI users such as MILPERSIS. Various aiter- 
na?i!e imo^ovements could be modeled and costed and the most cost-effective 
aSa Te dent fieS A generic CMI simulation model is currently being 
dSpeJ which may be used to support studies of this type for system 
expansion. 

Second the student load on the present system could be carefully and 
aradually increasenn.til significant performance degradation becomes appar- 
ent There is at present no reliable way, either objectively or subjectively, 
tS determine at what level of student loading such performance degradat on 
will oeLr. However, as performance- degradation occurs,, as it inevitably 
w co??;ctive act on can be taken to maintain system performance.. Such 
cJrr^ct Jelc ons must be technically .feasible and shou d be econom cally 
efficient. There are indeed certain risks associated with the continued 
pynansion of the Present CMI system, but these risks appear minimal. The 
ZlTXJs ?ob?e"Sould be L.plete and P-l-g«^,^y????/^;^^[^;;,rn 
Z ^^^rirlmSLf if oJ^f^^a^i g ^t rrncfelsr^n^^esp^nse^tL. 
I' ea a nc^'asf oSlr'the' existing response\ime of two to three seconds 
could be tolerated without having a significant J^Pact on train ng time 
sincp system specifications call for a response time of 30 seconds or less. 
?Ms'woJlld indicate that current response time could be increased without 
system specifications being exceeded. 

It is inevitable that at some point computer downtime will affect t^ 
inn time However, the present study was concerned with both duration ot 
Zn le'as wel a the frequency of interruptions and to what extent these 
SavfirpaaeS oJ training time. Available data does not suggest a direc 
one to one relationship between computer downtime and length of student 
time in train ng. In fact, the data available suggests that short (30 
ffl nutes 0^ ess) and relat vely infrequent downtimes have a minimal impact 
;j stSdeSt ear ing for a CMI System. Student time to mastery ^s. related 
To time spent in fulfilling certain specified conditions of earning. Most 
of.the sSf s studying and learning experiences in fulfilling those 
Conditions do not invJlve the computer. Computer downtime could only affect 
JhP Parnina rate of those students who demand service and the number of 
TtSdentTJlnding seJJice will depend on the length of downtime and frequency 
of student interaction with the computer. 
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Most learning centers do not have a complete manual back-up system. 
However, where such systems exist,, they appear to be very effective in mini- 
mizing negative effects of computer downtime. Properly designed back-up 
systems could be developed which would be effective in managing prolonged 
downtime. The high availability of the CMI systems~makes it questionable 
whether the development and maintenance of full-scale back-up systems would 
be cost effective. Most of the school staff consulted during this 'study 
were of the opinion that there was no measurable impact on training time 
from the relatively minor interruptions which have occurred during the latter 
part of the time period of this study. However, staff estimates of how 
long the computer must be down before there would be a significant exten- 
sion of training time varied from 15 minutes to 1 hour. From the above 
analysis it is estimated that the computer would, in most cases, need to be 
down for an hour or more before training time would be extended significantly. 
Between March and October 1981 there were relatively few instances where 
the CMI service to a course was down for an hour or more during the entire 
shift. Approximatley 95 percent of the interruptions lasted 30 minutes or 
less. 

It is reasonable to assume that a degree of distributed processing should 
be considered for any major expansion of the current CMI system. Possible 
advantages of distributed processing include elimination of the need for 
communication lines to process student transactions, the improvement of 
local conmand control and service, and the provision of redundancy where 
possible. Disadvantages include the potential loss of central control, 
possible scale diseconomies, and increased difficulty in maintaining soft- 
ware and hardware standardization. The cost effectiveness of distributed 
processing will depend, in part, on the costs of upgrading the existing 
system as well as development and implementation costs of a distributed 
system. More data needs to be collected to determine the requirements and, 
costs of upgrading the present system. Simulation would provide a basis 
for obtaining these data. At this time, there is not an acute need for a 
major redesign of the present system. Subject to a rapid and unexpected 
increase in courses placed in the CMI system, time is available to make a 
complete assessment of system need and to formulate a conceptual system 
design prior to any new development and Implementation. Additionally, proto- 
type implementation and evaluation are recommended before considering wide- 
scale application. A site phasing implementation plan should be^-developed 
which would assure training continuity during distributed system integration. 

As stated previously, the existing system is a good one and it can be 
expanded. Every possible step should be taken to assure that a replacement 
system will perform more efficiently. It should also be noted that a replace- 
ment system will form the basis for Navy computer based instructional manage- 
ment during the next decade. If the concept formulation phase of this develop 
ment does not include life cycle cost benefit assessment and if state-of- 
the-art network/coimiunications/data base architectures are not considered, 
there is a high probability of serious consequences for Navy training. The 
training community may be forced to live with a deficient system. 

Distributed processing is not a fixed approach to data processing but 
can encompass a wide range of software and hardware configurations. Dis- 
tributed processing also requires decisions about which functions can most 
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effectively or efficiently be processed remotely at the school rather than 
at a centrally located facility. Consequently, there can be a large number 
of alternative systems defined as distributed processing which will be cap- 
able of providing the required future CMI services. A number of the tech- 
nically feasible alternatives should be evaluated in order that the most 
cost-effective system can be identified for implementation. 

A final and obvious conclusion is that if the CMI processing requirements 
continue to increase, the CMI system will eventually be overloaded and require 
upgrading. At least two important areas must be addressed to determine the 
most cost-effective way to upgrade the system. First, the maximum efficient 
capability of the present system is unknown. Presently, there is no reliable 
way to identify the potential difficulties which will be encountered with 
increased processing requirements; therefore, there is no reliable way to 
determine the marginal cost of upgrading the present system. Simulation or 
future experience gained from increasing the load on the present system 
will eventually -provide answers to this problem. Second, at what point the 
processing requirements will exceed the capability of the present system 
depends on the rate and processing requirements of new courses brought under 
CMI. The processing requirements may not increase as rapidly for group- 
paced courses requiring only testing support as they would for self-paced 
courses. 

RECOmENDATIONS 

1. Develop a comprehensive simulation model which will provide a capa- 
bility for determining "bottlenecks" in the present system and for evaluating 
alternative CHI expansion strategies. Such a model would provide insight 
into the efficiency and effectiveness of alternative expansion strategies. 

2. Develop, implement, and evaluate a prototype distributed processing 
system. Such a system should be ready for operational implementation when 
and if it becomes noneconomical to further expand the present system. 

3. Using data from a planned CMI course implementation schedule, the 
simulation loodel, results from the prototype distributing system, and the 
technical capabilities of microcomputer technology, develop a long-range 
plan for system expansion. Options should include expanding the present 
CMI system, implementing distributed processing, and viable options which 
utilize both approaches. 

4. Develop a workable strategy for managing the students during computer 
downtime with emphasis on short interval interruptions. 

5. Because of high system reliability, reevaluate the cost-effective- 
ness of existing requirements for developing and maintaining comprehensive 
manual back-up systems to manage students during the longer downtimes. 
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Figure A-1. Histogram of Mean Response Times by Location 
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Figure A-4. Total Interruptions by Day of Week 
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Figure A-5. Histogram of Mean Downtime by Day of Week 
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Figure A-6. Total Time System Reported Down 
for Each Day of Week 
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Figure A-7. Total Time System Reported Down 
for Each Course 
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Figure A-8. Total Time System Reported Down 
for Each Location 
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Figure A-10. Mean Percentage of Time System Not 
Available by Day of Week 
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